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Abstract 

This paper considers the power allocation of a single-link wireless communication with joint energy 
harvesting and grid power supply. We formulate the problem as minimizing the grid power consumption 
with random energy and data arrival, and analyze the structure of the optimal power allocation policy in 
some special cases. For the case that all the packets are arrived before transmission, it is a dual problem 

m 

of throughput maximization, and the optimal solution is found by the two-stage water filling (WF) policy, 
which allocates the harvested energy in the first stage, and then allocates the power grid energy in the 

X 

second stage. For the random data arrival case, we first assume grid energy or harvested energy supply 
only, and then combine the results to obtain the optimal structure of the coexisting system. Specifically, 
the reverse multi-stage WF policy is proposed to achieve the optimal power allocation when the battery 
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capacity is infinite. Finally, some heuristic online schemes are proposed, of which the performance is 
evaluated by numerical simulations. 

Index Terms 

Energy harvesting and power grid coexisting, convex optimization, two-stage water filling, reverse 
multi-stage water filling. 

I. Introduction 

Minimizing the network energy consumption has become one of the key design requirements 
of wireless communication systems in recent years [fl], A3- By exploiting the renewable energy 
such as solar power, wind power and so on, energy harvesting technology can efficiently reduce 
CO2 emissions Q. It can greatly reduce the load of power grid when the renewable energy 
usage well meets the network traffic load. However, due to the randomness of the renewable 
energy, users' quality of service (QoS) may not be guaranteed all the time, which requires other 
complementary stable power supplies. As the power grid is capable of providing persistent power 
input, the coexistence of energy harvesting and grid power supply is considered as a promising 
technology to tackle the problem of simultaneously guaranteeing the users' QoS and minimizing 
the power grid energy consumption [0J. 

A number of research paper have focused on the energy harvesting issues recently. The 
online power allocation policies of energy harvesting transmitter with a rechargeable battery are 
studied in [21-[[8]| using Markov decision process (MDP) approach (51 . The energy allocation 
and admission control problem is studied in for communications satellites. Ref. [6J considers 
the cross-layer resource allocation problem to maximize the total system utility. In 0, the 
authors develop the energy management policies with stability guarantee in sensor networks. 
And throughput maximization with causal side information and that with full side information 
are both studied in [8|. Although some optimal online policies are proposed, the structure of 
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the optimal power allocation is still not clearly presented via MDP approach, which in addition 
usually suffers from the curse of dimensionality Q. Recently, there have been research efforts 
on the analysis of the optimal offline power allocation structure lfT0l - lfl2l . In ifTOl . the problem 
of minimizing the transmission completion time with infinite battery capacity in non-fading 
channel is studied in two scenarios, i.e., all packets are ready before transmission and packets 
arrive during transmission. Ref. [UTI finds the optimal transmission policy to maximize the 
short-term throughput with limited energy storage capacity, and exploits the relation between 
the throughput maximization and the transmission completion time minimization. The power 
allocation of energy harvesting systems in fading channel is formulated as a convex optimization 
problem in [fT2l . and the optimal directional water-filling (WF) policy is introduced. 

Besides the analysis of energy harvesting system listed above, there is some other work 
on offline power allocation, which optimizes the energy efficiency assuming that the required 
energy is always available (e.g. power grid). In |[T3l . a lazy packet scheduling policy is proved 
optimal to minimize the total energy consumption within a deadline. The similar problem with 
individual packet delay constraint is studied in |[T4l . The authors in |fT51 considered the strict 
QoS constraints such as individual packet deadlines, finite buffer and so on, and proposed a 
general calculus approach. Different from the existing work, we consider the power allocation 
problem in the wireless fading channel with the coexistence of energy harvesting and power 
grid. To the best of our knowledge, very limited work such as [H has focused on this topic, 
which however, only considers the binary power allocation. 

In particular, we study the offline power allocation problem aiming to find the optimal structure 
in the energy harvesting and power grid coexisting system. We assume that the data is randomly 
arrived in each transmission frame, and minimize the average power grid energy consumption 
while completing the required data transmission before a given deadline. The main contributions 
are presented as follows: 

• The offline grid power minimization problem is formulated and then converted into a 
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convex optimization problem. Then the closed-form optimal power is obtained in terms 
of Lagrangian multipliers. 

• The structure of the optimal power allocation is analyzed in some special cases. Specifically, 
in the case that all the data is ready before transmission, the problem is a dual problem 
of the throughput maximization and is solved by the two-stage WF algorithm. In the case 
that the battery capacity is infinite, the optimal water levels are non-decreasing, and the 
optimal power allocation structure can be obtained by the reverse multi-stage WF policy, 
which allocates the harvested energy and the grid energy one by one from the last frame 
to the beginning. 

• Some sub-optimal online schemes based on the analysis of the optimal power allocation 
structure, including constant water level policy and adaptive water level policy, are proposed 
and evaluated by numerical simulations. It is shown that the adaptive water level policy 
performs better than the constant one considering both the grid energy consumption and 
the QoS guarantee. 

The rest of the paper is organized as follows. Section [I] introduces the model of the energy 
harvesting and power grid coexisting system. The offline grid power minimization problem is 
formulated in Section UTTJ In Section [TV] we study the optimal solution for the minimization 
problem by cases. Then some online algorithms are proposed in Section |Vj and numerical 
results are presented in Section |VI] Finally, Section I VII I concludes the paper. 

II. System Model 

We consider a single wireless link as shown in Fig. \T\ where the transmitter is powered by 
both energy harvesting and power grid. The harvested energy is stored in a battery with capacity 
E m ax for data transmission usage. If the energy in the battery is insufficient, the transmitter uses 
the power grid to guarantee the user's QoS. In this paper, we assume the energy is used only 
for transmission, i.e, the processing energy is ignored. 
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Fig. 1. Single-link wireless communication model with the coexistence of energy harvesting and power grid. 



Assume the system is slotted with frame length T/. We study the optimal power allocation 
during a finite number of frames N, indexed by {1, . . . , N}. We adopt the block fading model, 
i.e, the channel keeps constant during each frame, but varies from frame to frame. The received 
signal in frame i e {1, . . . , N} is given by 

y% = hiy/plxi + rii, (1) 

where hi is the channel gain, p, L is the transmit power, Xi is the input symbol with unit norm, 
and rii is the additive white Gaussian noise (AWGN) with zero mean and unit variance. The 
reference signal-to-noise ratio (SNR), defined as the SNR with unit transmit power, during the 
frame i is denoted by ji = \hi\ 2 . Then, the instantaneous rate in bits per channel use is 

r i = ^log 2 (l +liPi)- (2) 

Assume the initial battery energy is E , and the harvested energy i = 1, . . . , N — 1 arrives 
at the beginning of each frame i. Part of the transmit power pi is supplied by the battery, denoted 
as pn,i- And the rest is supplied by the power grid pa,i = Pi~PH,i- The frame and energy arrival 
model is depicted in Fig. [21 
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Fig. 2. Frame model and the energy arrival and storage structure. 



There are several constraints on the battery power p H i : Since the harvested energy cannot be 
consumed before its arrival, the first constraint is the energy causality 

k fc-1 

Y, T fPH,i<Y< E " k = l,2,...,N. (3) 

i=l i=0 

The second constraint is the limited battery capacity. The battery overflow happens when the 
reserved energy plus the harvested energy exceeds the battery capacity, which however, is not 
preferred because the data rate can be increased if the energy is used in advance instead of 

overflowed. So we have 

k k 

J2 E i~J2 T fP H ^ E ^ k = l,2,...,N-l. (4) 

i=0 i=l 

On the other hand, the grid power satisfies the average power constraint 

N „ N 



(5) 



Jt ^PG,i = ^2(Pi ~ PH,i) < ^G,ave, 
i=l i=l 

where Pc,avc is the average grid power supply. 
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III. Grid Power Minimization Problem 

We study the grid power minimization problem with offline setup, i.e., the energy arrivals E { 
and the channel gains 7, of all the frames are known in advance. Assume that the amount of B{ 
bits arrive at the beginning of frame i + 1, where z = 0,l,...,iV — 1, and all the quantities are 
known. The constraint due to the random packet arrival is that the number of transmitted bits 
can not exceed those in the buffer, which can be expressed as 

^^log 2 (l + 7iPi )<^ J B i , k = l,...,N-l (6) 

i=l i=0 
N rp N-1 

^^log 2 (l + 7^.) = ^^. (7) 

i=l i=0 

By using © minus © for all k = 1, . . . , iV — 1, and then relaxing © to inequality, we get the 
data constraint as 

N T N-1 

£)-^log 2 (l + 7iPi)> J2 B " k = l,...,N (8) 

i=k i=k—l 

which is equivalent with the original constraints © and ^7} because while achieving the opti- 
mality, ([8]) for k = 1 must be satisfied with equality. Otherwise, we can always decrease the 
transmission rate by reducing the energy consumption without conflicting any other constraints. 
In addition, the relaxation of © avoids the conflicts between power and data constraints. For 
instance, if E — E% — -E max and B — 0, the original constraint © conflicts with the battery 
constraint © for k = 1. In this case, the problem is infeasible. However, the problem with 
constraint © is always feasible since we can use the extra harvested energy to transmit packets 
carrying no data to avoid battery overflow. In the example, the feasible power allocation policy 
is to allocate E to frame 1 to transmit packets without any data. 

We aim at minimizing the energy consumption of power grid under the constraints of re- 
quired data transmission, causality of harvested energy, battery capacity and non-negative power 
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allocation. The problem is formulated as follows 

N 



mm 

Pi,PH,i 

i=l,...JV t=l 



J2 T f(Pi~PH,i) (9) 

t=l 

s.t. ©,©, and ©, 

Ret,* < Pk, Vfc (10) 

Pjt,*>0, VA; (11) 

where the constraints (fTOl and ([Til imply that the energy from either power grid or battery is 
non-negative. Notice that there are three cases of the solutions: (a) The harvested energy exceeds 
the required energy to transmit all the arrived packets, which results in pi = p Hi , and there are 
multiple solutions; (b) The harvested energy is insufficient to transmit the packets. Then we 
have pi > p H>i , and the inequality is satisfied for at least one i; (c) The harvested energy is 
just enough to achieve the optimal power allocation, and any additional packets to transmit will 
result in non-zero grid power. In our analysis, we mainly focus on the case (b) where grid power 
is necessary. The case (c) is also studied as it helps us to understand the structure of the optimal 
harvested power allocation. 

Also note that we ignore the average grid power constraint ©, which decides the feasibility 
of the problem. If we can find a feasible solution for problem © without violating ©, it can 
be ignored. Otherwise, the required bits can not be transmitted by the deadline. Hence, the best 
we can do is try to maximize the number of transmitted bits, which can be formulated as a 
throughput maximization problem 

N T 
i=l,...,JV i=l 

s.t. (ED,©,©, (HUD, and (P). 

In the following section, we analyze the optimal solution for the grid power minimization problem 
assuming it is feasible (the constraint © is ignored). Also, the relation between the throughput 
maximization and the grid power minimization is briefly discussed. 
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IV. Offline Optimal Solution Structure 

The objective © is a linear function, the constraints © are convex for all k since it is a sum 
of log functions larger than a threshold, and the others are all linear constraints. Consequently, 
the optimization problem is a convex optimization problem, and the optimal solution satisfies the 
KKT conditions fT6ll . Define the Lagrangian function for any multipliers > 0,/ifc > 0,r] k > 
0, a k > 0, k > as 

N 



£ = ^2 T f(Pi - PH,i) 
i=l 

N / k k-l 

+E A ME T /^-E^ 



k=l \i=l i=0 
N-1 / k k 



+ e [ e E i — e t iph,i - e 

m; 

k=l \i=0 i=l 

N / N rp N 

-E^ Ey 1o S2(! + TiPi) - E 5 « 

k=l \i=k i=k—l 
N N 

+ 22 a k(pn,k - Pk) - 2J ^mfc (13) 



fc=l k=l 

with additional complementary slackness conditions 

k k-l 



x k E - E Ei ) = °> VA; (14) 

\i=l i=0 J 

(k k \ 

J2 E >~J2 T fP^ - ^max = 0, k < N (15) 
i=0 i=l J 
f N N-1 \ 

E y log2(1 + liPi) - E ^ ) = °- VA; < 16 ) 

V i=k i=k—l J 

ak{PH,k ~Pk) = 0, Wk (17) 

/3kPH,k = 0, VA; (18) 

We apply the KKT optimality conditions to the Lagrangian function (fl"3l) . By setting dC/dpi = 
dC/dpn,i = 0, we obtain the unique optimal power levels p* in terms of the Lagrange multipliers 
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as 

* = Se^-I, (19) 

l-ai li 

ovp* = N ^ k = lVk -, (20) 

where /i^ = 0. Notice that there is no closed-form expression for the parameter pj^, which 
indicates that the power supplied from battery can be of many choices as long as the constraints 
are satisfied and the optimal total power p* is achieved. For instance, if one of the optimal power 
allocation for frames i and i + 1 is Pi = p i+ \ = 2,p Hji = l.5,pn,i+i = 0.5, then it is also optimal 
by allocating p H ,i = Ph,i+i = 1- Denote p* Hi a.s a. feasible optimal solution in order to distinguish 
from the optimization parameter p H>i . Hence, the optimal power allocation is 

1 



u t 



(21) 



Pi 

where [x] + = max{i, 0}, and the water level is expressed by either 

i 

Vi = ^Vk, (22) 

k=l 

OXVl = p k=1 Vk , (23) 

depending on how the battery power and the grid power are allocated. 

Based on the optimal power allocation solution (|2Tb -(l23l. we study the following special cases 
to find the structure of the optimal power allocation. 

A. Packets Ready Before Transmission 

Assume the packet arrival sequence is {B, 0, . . . , 0}, i.e., all the packets are ready before 
transmission. In this case, the optimal power allocation is simplified as 

Pi = rr^ — , (24) 

l-ai li 



Vi l_ 



orp* = ^ N - n ( 25 ) 
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We have the following proposition: 

Proposition 1. For given harvested energy profile Eh^, i = 1, . . . , N, define /(PG,ave) = max 
YliLi ~2 k>g(l+7iPi) under the constraints ©, (0), ©, rfTOl ), and 4771) . Asswme ?/ze packet arrival 
sequence is {B, 0, . . . , 0}, and define g{B) = min i 5_)i=i ^!f(P« — Pff.i) under the constraints 
®, ®, ©, ((70]), and ([77]). Wfe /*ave / = g~\ 

Proof: First of all, the functions / and g are monotonic increasing since the log functions 
are monotonically increasing and concave. 

Then, we set B max = /(P),P min = g(B max ). If P min > P, we have f(P min ) > f(P) = 
B max . Hence, g(B max ) < P min , as with P min , we can achieve higher throughput /(P min ) by the 
optimal policy. It contradicts the assumption. The same result holds for P min < P. As a result, 



Proposition \T\ shows that if all the packets are already at the transmitter before the transmission 
starts, the problem © is a dual problem of (fT2l> . We now discuss the structure of the optimal 
power allocation of problem (TT2T ). and our grid power minimization problem can be solved in the 
same way due to dual property. In fact, the problem (fl"2T ) can be considered as a simple extension 
of [fT2ll by adding an additional average grid power constraint. Hence, it can be solved similarly 
by the directional WF algorithm. In particular, the harvested energy in each frame can only be 
transferred to the future frames, and the amount that can be transferred is limited by the battery 
capacity. It is realized by introducing the concept of right permeable tap [12]. The difference 
in our problem is that there is additional grid energy, which however, can be considered as an 
additional battery with total amount of energy NPo^Tf. As there are no causality and capacity 
constraints, the grid energy allocation can be viewed as the directional WF from frame 1 with 
right permeable tap which allows any amount of energy transfer, which is in fact the traditional 
WF algorithm [fTTl. 

In order to emphasize the difference between harvested energy and grid energy as well as 
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Fig. 3. Example of two-stage WF operation, (a) shows the harvested energy arrival; (b) shows the procedure and the result of 
directional WF; and (c) shows the two-stage WF result. 



motivate the algorithm design for random packet arrival case, we propose the two-stage WF 
optimal power allocation policy. The procedure is divided into two stages. In the first stage, the 
system operates directional WF [|T2l to allocate the harvested energy. At the beginning of the 
second stage, the surface of the water freezes up. Then the traditional WF [17] is performed to 
allocate the power grid energy on the ice surface, until the energy is used up. 

An example is shown in Fig. [3j We illustrate a total of N = 12 frames. The initial battery 
energy is E , and only the values of harvested energy E5, E 7 and E w is non-zero. By performing 
the two-stage WF algorithm, the optimal power allocation is obtained. It is observed from the 
figure that a constant water level is achieved except for frames 3, 6 and 7. The power used in 
frame 3 is zero because of deep fading. Due to the battery capacity constraint, frames 6 and 7 
have higher water level to avoid energy overflow. Also, the power allocation of harvested energy 
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and power grid energy is variable under the water level. For instance, the harvested energy 
allocated in frame 2 can be partially reserved to be used in frame 4, and the water level in frame 
2 is achieved by power grid energy. 

B. Random Packet Arrival 

Now assume the packets arrive in every frame. In this condition, we first analyze the structure 
of the optimal power allocation under the different assumptions on the energy. Then based on 
the observations, we discuss the power allocation for the general case. 

Proposition 2. If Ei = for all i = 0, 1, . . . , N — 1, the optimal water levels are monotonically 
non-decreasing as Vk < ^k+i- The water level may increase only when © is satisfied with 
equality. 



Proof: Without the energy harvesting, the water levels are expressed as (1221) . Since r] k+ i > 0, 
it naturally results in v k < fk+i- In addition, if © is satisfied without equality, i.e., 

^-llog 2 (l + 7iPi ) <Y,Bi, (26) 

i=l i=0 



N N-1 
1 f 



from © we get 

ylog 2 (l + 7iPi) > J2 B >- ^ 

i=k+l i=k 

Then from (fT6l we have r/k+i = 0, and hence v\. = Vk+i- ■ 
Note that the energy minimization by a deadline with grid power supply only is studied in 
|fT3l for AWGN channel, where the rate-power function is time -invariant. Here, we consider the 
fading channel case, and conclude from Proposition |2] that if the power is supplied by the power 
grid only, the water level is non-decreasing from frame to frame. It increases only at the points 
where 

^^log 2 (l + 7 ^) = E 5 - (28) 



2 

t=l i=Q 
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i.e., the already arrived packets are all transmitted by the end of frame k. The optimal power 
allocation can be obtained by reverse multi-stage WF algorithm, which allocates the power from 
the last frame to the first frame. First allocate the power to frame N so that © for k = N is 
satisfied with equality. Then freeze up the water surface of frame N, and allocate the power to 
frames N — l and N with the traditional WF policy on the ice surface so that d8) for k = N — 1 is 
satisfied with equality. Repeat the procedure until the first frame. We summarize it as Algorithm 

D 



Algorithm 1 Reverse multi-stage WF with grid power 



1: 


Set 7j = 7 ia i = 1, . . 


.,iV. 








2: 


for all k = N to 1 do 








3: 


Find v k so that 














AT rr 

El'* 

i=k 


5 2 (1 +7iPi) = B k _!,pi = 


1 

^fc - — 
7*. 


+ 

(29) 


4: 


Update 7j as 














li 


= \ Pi + ^-j ,i = k,... 


,iV. 


(30) 


5: 


end for 










6: 


p? = ^ - -,i = 1, 


..,N. 









In the algorithm, for a given fc, 7^ denotes the freezed water level updated in the previous 
stage i — l,i = k + 1, . . . N, and 7^ = 7^ as was set in step 1. Hence, (1291 means that in this 
stage, Bk-i bits arrived at the beginning of frame k are served by the traditional water filling 
algorithm in the following frames (from k to N). Fig. |4] demonstrates an example of reverse 
multi-stage WF with grid power. It can be seen the arrived bits B 2 and B 3 are served through 
frames 3-5, resulting in the same water level in these frames. The rest bits are served in their 
arriving frames respectively. 
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Fig. 4. Example of reverse multi-stage WF with grid power. Starting from the last frame to the first one, Bi is served in frames 
i,i + 1, . . . , N with traditional WF scheme. 



Proposition 3. If E max = oo, p* = p* Hi for all i = 1,2, . . . , N, and p* Hi is unique (i.e., 
any packet arrival set {B' Q , . . . , -B^r-i} satisfying B' t > B it and J^^Jq 1 B[ > Yld=o B% results 
in YuiLiTfip* —P*Hi) > ®)> tne optimal water levels are monotonically non-decreasing as 
Vk < Vk+i. The water level may increase when either (0) or © is satisfied with equality. 

Proof: Under the conditions that the energy battery has infinite capacity and there is a 
unique harvested power allocation policy to complete the transmission without grid power input, 
the water level is determined by 



Ek 
3=1 VS 

J2f=k ^3 



Vk = ^ r ■ (3D 



Since r] k+1 > 0, > 0, we have u k < u k+1 . 

The water level increases when either r/k+i > or \ k > 0. Based on (fl4l) . Xj. > indicates 
that © is satisfied with equality. Based on © and ([TBI , i] k+1 > indicates that © is satisfied 
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with equality. ■ 
Proposition [3] shows the structure of optimal harvested power allocation under infinite battery 
capacity constraint. The conclusion is an extension of the optimal power allocation in non-fading 
scenario (Lemma 6 in IfTOlO to the fading scenario. The intuition is that due to the causality of 
energy and data arrival, the later frames can "see" more available energy to be used and data bits 
to be transmitted than the earlier frames as the energy and data can be transferred rightwards, 
rather than leftwards. Hence, the water level is non-decreasing. Without the battery capacity 
constraint, the harvested energy can be reserved for future use in any way as one wishes, and 
the arrived packets can also be transmitted in future frames . In addition, the water level changes 
when either all the packets arrived before the current frame are transmitted or all the energy 
harvested before the current frame are used up. If both the packet buffer and the energy battery 
are not empty, the water level keeps constant. 

Proposition 4. If E m3uX = oo, the optimal water levels are monotonically non-decreasing as 
Vk < ffc+i- In addition, there is an optimal policy where a unique k exists such that for k < k, 
Uk is expressed as < [22\) and for k>k,Vk is expressed as 071) . 

Proof: We prove the proposition by studying all the cases of power allocation between 
energy harvesting and power grid in two consecutive frames. 

(a) If p* k > p* Hk and p* k+1 > p* Hk+1 , the water levels of frame k and k + 1 have the same 
expression as (l22l) . hence Vk < i^k+i- 

(b) If p* k = p* Hk and p* k+1 = p* H k+1 , the water levels are expressed the same as (|3TT i. We also 
obtain the monotonicity. 

(c) If p* k > p* H k and p* k+l = p* H>k+1 , we have a k = 0, a k+l > 0. As a result, v k = Ej=i Vj < 

(d) If p* k = p* Hk and p* k+1 > p* Hk+1 , we can exchange the amount of harvested energy 
Tf mm{p* H k , p* k+1 —p* Hk+1 } of frame k with the same amount of grid energy of frame k+ 1 to 
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achieve the same optimal result, without conflicting any constraint or changing any water level. 
Consequently, this case is converted to either (a) or (c), and the monotonicity holds as well. 

To sum up, Uk < Vk+i holds for all k. In addition, by conservatively using the harvested 
energy, the case (d) is converted to (a) or (c), hence does not exist. Depending on (a)-(c), there 
is a unique k such that for k < k, we have p* k > p* Hk and p* k+1 > p* H k+1 , for k > k, we 
have pi = p* Hk and p* k+1 = p* H>k+1 , and for k = k, we have p* k > p* H k and p* k+1 = p* Htk+1 . 
Equivalently, for k <k, u k is expressed as (TUT) and for k > k, v k is expressed as (13TI) . ■ 

Proposition |4] presents an analytically tractable structure of the optimal power allocation for 
both energy harvesting and power grid. Intuitively, the harvested energy is reserved in the battery 
for the use in the later frames, in order to reduce the effect of causality constraint and improve 
the flexibility of harvested power allocation. Hence, the harvested power and the grid power are 
allocated one by one with the structures described in Proposition [2] and [31 respectively. 

The optimal water level can be obtained by the power allocation policy which avoids case 
(d) in the proof of Proposition |4] It is structured as follows: The water level is non-decreasing 
and the harvested energy is used in a conservative way. That is, the energy harvested before 
frame k is saved for future use so that the required transmission after frame k can be satisfied 
by harvested energy only. To minimize the power grid energy, the harvested energy should be 
allocated to maximize the achievable throughput. According to Propositions |3] and @] the optimal 
harvested energy allocation takes the similar reverse calculation structure as Algorithm Q] The 
difference is that we should consider the causality of energy arrival. As the harvested energy 
cannot be used before its arrival, it should be fully utilized after arrival. Specifically, denote the 
available power as pe,n = E^-i/Tf, and the required power as pb,n = {^? Bn ^I t s — \)/^ N . 
If Pe,n < Pb,n, i-e-, the harvested energy in frame iV is not enough to transmit B N ^i bits, we 
"move" the energy from the previous frames until B^-i bits can be transmitted. If pe,n > Pb,n, 
i.e., Bn-i bits transmission does not use up the harvested energy E^-i, we move the packets 
from the previous frames until E N ^i is used up. Then we freeze up the water surface and allocate 
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power to the frames N — 1 and iV with the traditional WF policy to achieve the same result, 
i.e., either the bits arrived in this frame are served or the energy arrived is used up. Repeat the 
procedure until the harvested energy is used up. It is detailed as Algorithm |2] 



Algorithm 2 Reverse multi-stage WF with harvested energy 



l: Set 7i = 7i, B { = Bi, E { = E h Mi. 
2: for all k = N to 1 do 



3: 

4: 
5: 
6: 

7: 



10: 
11: 
12: 
13: 
14: 
15: 
16: 



N 



Find u Ejk so that J] T fPE,i = E k -i,p E>i 

i=k 



k — ~ 



N 



Find u B ,k so that J2 ~k log 2 (l + 7*Pb,») = B k _ u p B . 



if u E ,k > VB,k then 
Find j* = max < j 



k-l 



N 



E B i > E -2 1 °S2( 1 + liPE,i 
i=j i= k 



k-l ... 

Update %i>k as ([H, p { = p Eji , B r = 5 i~E ~i ]o S2Q-+l<PE,i), Bi = 0,i> f 

i=j* 

else 

Find j* = max j 
if {j*} = then 



N 



k-l _ N 

EEi>E T fP B,i 

i=j i=k 



Find {p-}i> fc = arg max -f log 2 (l + jiPi). 
Update %,i > k as d30l) with p« = p^, then break, 
else 

fe-l _ N 

Update %i>k as CEB with p, = p B>i , E r = J2 E i ~ E t /Pb,^ #i = 0, i > j*. 
end if 
end if 



17: end for 

18: P*HA 



Ti 7« ' 



AT. 



Note that we assume the harvested energy is not enough to satisfy the required transmission, a 
proper j* can always be found in step 6, i.e., we can shift the arrived bits from previous frames 
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to the current one to use up the arrived energy. On the contrary, j* in step 9 may not be found 
when k <k, and Algorithm |2] terminates. Then the water surface freezes up and the grid power 
is allocated using Algorithm \T\ on the ice surface until all the bits are transmitted. Consequently, 
the optimal water level is found. 

In summary, the power allocation for infinite battery capacity is done in two stages. In the 
first stage, harvested energy is allocated reversely from last frame by the reverse multi-stage 
WF algorithm with harvested energy. We call the stage in this algorithm as sub-stage. In each 
sub-stage, the to-be-transmitted bits form a required water level, and the to-be-used energy can 
achieve an available water level. If the required water level is higher than the available water level, 
energy from earlier frames are transferred rightwards. Otherwise, the bits from earlier frames are 
transferred. The sub-stage ends when the two water levels are equal. When the harvested power 
allocation stage is finished, the water surface freezes up and the grid energy is allocated in the 
second stage also in multiple sub-stages. The objective of each sub-stage is just to achieve the 
required water level. 

However, in general case, if the battery capacity is finite, the monotonicity of the optimal 
water level does not hold when the constraint © is satisfied with equality. The structure of the 
optimal power allocation cannot be described in a simple and clear way. Intuitively, in contrast to 
the case of infinite battery capacity where the harvested energy is conservatively used to achieve 
the optimality, in the finite capacity case, the power from battery should be allocated first to 
minimize the energy waste. We will propose some online algorithms based on the intuition. 

V. Online Power Allocation 

In this section, we assume that the transmitter only has the knowledge of the battery energy 
levels, the energy arrivals and the channel states of the current frame and all the past ones, 
but does not know those of the future. Besides, the statistics of energy arrival and fading are 
available to the transmitter. In this setup, some online power allocation algorithms are proposed. 
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A. All packets ready before transmission 

If all the packets B are already at the transmitter before transmission, such as file transfer, 
the following algorithms are proposed: 

1 ) Constant water level: Motivated by the offline optimal solution that the power allocation 
tries to achieve a constant water level (by the traditional WF in the second stage), we can propose 
a constant water level algorithm. Assume the constant water level is I/70, which is calculated 
by solving the following equation 



The power for each frame % is calculated as 

Pi = \- ~ -1 • (33) 

.7o 7*. 

If TfPi < E Q:i , where E Q:i denotes the battery energy at the beginning of frame i, the required 
power is supplied by the battery only, and the energy from power grid is zero, i.e, Pi = p H ^. 
Otherwise, the amount of energy Tjp { — Eq^ will be taken from power grid. 

2) Adaptive water level: For the finite time transmission, the constant water level algorithm is 
apparently not optimal due to the different realization of channel gains. We propose the adaptive 
water level algorithm to improve the performance. The water level is updated for each frame, 
denoted by 7 0)i , which can be obtained by solving 



where B R i is the remaining bits at the beginning of frame i. 

Due to the randomness of energy arrival, there may be energy battery overflow for online 
solutions. Hence in addition, we propose an energy overflow protection algorithm. If the expected 
battery energy exceeds the battery capacity, i.e, E Q:i + Pff iave T/ > £ max , a minimum power 




(32) 




(34) 



Pmin 




max 



H,&ve 



(35) 
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must be used. Then the power allocated in frame i is determined as 



Pi = max{pi,p min }, 



(36) 



where pi is expressed as (|33T > . In this way, a certain amount of power must be used even when 
the channel is in deep fading or the water level exceeds the optimal one. This algorithm well 
meets the battery capacity constraint from the average point of view, and is expected to improve 
the performance, although the battery overflow can not be completely avoided. 

Notice that all the packets need to be delivered in N frames. Hence, in the last frame, all the 
remaining packets should be transmitted as long as the required transmit power is achievable. 
Specifically, the transmit power of the last is determined by 



where Pr,max is the maximum achievable transmit power. However for the online settings, the 
transmission completion can not be guaranteed due to the fading, the maximum transmit power 
limits, and so on. We assume that the packets remained in the transmitter by the end of the last 
frame are dropped, and define the packet dropping probability as the ratio between the number 
of bits remained after the transmission and the total amount of bits arrived. Online algorithms 
should minimize the grid energy consumption as well as the packet dropping probability. 

B. Random packet arrival 

For the case that the bits arrive at the transmitter randomly, the online algorithms can be 
proposed as a direct extension of that for all packets ready before transmission case. Specifically, 
the constant water level algorithm is as follows. Assume the average packet arrival rate is B 
bits/frame, then the constant water level I/70 can be obtained by 




(37) 




(38) 
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The difference with the previous one is that, as the available number of bits is finite, the power 
used in frame i should not exceed 



I 7* J 

Hence the transmit power is expressed as pi = min {pi,p mSLX } , where Pi is expressed as (|33~T ). 
The adaptive water level can be calculated as 



where the right side of the equation is the total number of remaining bits approximated as the 
sum of current remaining bits B^i and expected arrival in the upcoming frames. 

The battery overflow protection algorithm needs to be modified as the number of available 
bits in each frame is random and finite. In particular, the transmit power is determined by 

p\ = min {max{pi, p min } , p max } , where p min is expressed as (|35T ). 



In this section, we test the performance of the proposed algorithms. We take the EARTH 
project simulation parameters for the femto-cell scenario lfT8l . Specifically, we set the maximum 
transmit power as 33dBm for simulations. A total number of 1000 monte carlo simulation runs 
are performed for each parameter setup. For each run, the channel reference SNRs and the 
energy arrivals are randomly generated. The distribution of reference SNR follows the Rayleigh 
model with probability density function p 7 (7i) = = exp ^— ^ j , where 7 is the average reference 
SNR (set to be OdB). The harvested energy follows non-negative uniform distribution with 
mean Pff. ave = 20dBm. The offline optimal solution, which is obtained by solving the convex 
optimization problem assuming all the reference SNRs and the energy arrivals are known before 
transmission, can be considered as the performance upper bound. 

We first exam the performance of the algorithms for the all packets ready before transmission 
case. We run iV = 100 frames in each simulation. The battery capacity is set M times of 
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Fig. 5. Performance of average grid power consumption versus number of bits per bandwidth when packets are ready before 
transmission. N — 100, Ph^vc = 20dBm, and the relative battery capacity M = 3. 



the average arrived energy in one frame, i.e, -Emax = MPH, avc Tf. The parameter M is called 
relative battery capacity. And we set M = 3 in this setup. The average grid power consumption 
achieved versus the number of bits to be transmitted is shown in Fig. [51 It shows that the 
adaptive water level algorithm outperforms the constant one. Because of the finite time length, 
the adaptive algorithm can better utilize the energy. Besides, the battery overflow protection 
algorithm improves the performance. It efficiently reduces the energy overflow at low grid power 
regime, but the gain shrinks as the grid power increases. 

We also investigate the influence of renewable energy battery capacity on the performance. 
Fig. [6] shows the relation between the battery capacity and the grid power consumption. Here, 
we set the number of bits per bandwidth as 25 bits/Hz. The performance gap between the online 
algorithm and the offline algorithm becomes small as the battery capacity increases. It shows 
that larger battery capacity makes the power allocation more flexible. Also, similar to the result 
from Fig. [5] the battery overflow protection algorithm is efficient at low battery capacity regime. 
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Fig. 6. Performance of average grid power consumption versus relative battery capacity when packets are ready before 
transmission. TV = 100, Ph,hvc = 20dBm, and the number of bits per bandwidth is 25 bits/Hz. 
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Fig. 7. Packet dropping probability versus number of bits per bandwidth when packets are ready before transmission. N = 
100, Pfl> vc = 20dBm, and the relative battery capacity M = 3. 
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Fig. 8. Performance of average grid power consumption versus number of frames when packets are ready before transmission. 
Ph,&vc = 20dBm, M = 3, and the number of bits is 0.25 bits/Hz/frame. 



Fig. [7] demonstrates the packet dropping probability for different packets transmission require- 
ments. It can be found that the adaptive algorithm can effectively complete the transmission for 
almost all the conditions (the packet dropping probability is less than 0.4%). On the contrary, 
the constant water level algorithm performs worse. Specifically, without overflow protection, the 
packet dropping probability keeps relatively high (> 4%), while with overflow protection, it is 
small for very few packets transmission requirement, and then increases and converges to that 
without overflow protection. Hence, the adaptive water level algorithm is an effective online 
algorithm. 

In addition, the average grid power consumption versus the number of considered frames iV 
is depicted in Fig. [8l Generally speaking, the grid power consumption becomes stable when the 
number of frames gets large. It can be seen that N > 100 is enough to get stable results, which 
validates our parameter setting. 

We then evaluate the performances of the algorithms for random packets arrival case. Figs. [9] 
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Fig. 9. Performance of average grid power consumption versus average data rate when packets randomly arrive during 
transmission. TV = 100, Ph,hvc = 20dBm, and the relative battery capacity M — 3. 
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Fig. 10. Performance of average grid power consumption versus relative battery capacity in grid power minimization problem 
when packets randomly arrive during transmission. N = 100, Ph, avc = 20dBm, and the average data rate is 0.25 bits/frame/Hz. 
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Fig. 11. Packet dropping probability versus average data rate when packets randomly arrive during transmission. N = 
100, Pfr.avc = 20dBm, and the relative battery capacity M = 3. 



and [10] show the grid power consumption versus average data arrival rate and relative battery 
capacity, respectively. In contrast to the previous results, we find that the constant water level 
algorithm with battery overflow protection achieves lower grid power consumption than the 
adaptive one. However, it is observed from Fig. [JT|that the constant water level algorithm results 
in much higher packet dropping probability (> 12%) compared with the adaptive algorithm 
(< 1%). The constant water level algorithm reduces the grid power consumption by dropping a 
large amount of arrived bits, which can not well meet the data transmission requirement. Hence, 
in random packets arrival scenario, the adaptive water level algorithm requires more grid energy 
than the constant water level algorithm in order to keep a low packet dropping probability. 

VII. Conclusion 

We have analyzed the optimal power allocation structure of the energy harvesting and power 
grid coexisting systems. We consider the power grid energy minimization problem and solve it 
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by convex optimization. The two-stage WF algorithm is proposed to solve the problem in the 
case that all packets are ready before transmission, and the multi-stage algorithm is proposed to 
solve it in the case of infinite battery capacity. In addition, online algorithms motivated by the 
offline solutions are also proposed and evaluated by numerical simulations. For all packets ready 
before transmission case, the adaptive water level algorithm outperforms the constant one for 
both grid power consumption and packet dropping probability. On the contrary, if the packets 
randomly arrive in each frame, constant water level algorithm achieves lower power consumption 
but causes much higher packet dropping probability. 
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